TST: Test ArrowExtensionArray with decimal types #50964
Conversation
|
[XPASS(strict)] No pyarrow kernel for decimal128(4, 3) |
| + [Decimal("-2.0"), Decimal("-1.0")] * 44 | ||
| + [None] | ||
| + [Decimal("0.5"), Decimal("33.123")] | ||
| ) |
There was a problem hiding this comment.
looks like if you pass Decimal("nan") to pyarrow it raises. is that going to be relevant for us?
There was a problem hiding this comment.
Looks like we coerce to NA
In [6]: pd.Series([decimal.Decimal("nan")], dtype=pd.ArrowDtype(pa.decimal128(4, 1)))
Out[6]:
0 <NA>
dtype: decimal128(4, 1)[pyarrow]
Which I guess is fuzzy with the NA vs nan debate
| request.node.add_marker( | ||
| pytest.mark.xfail( | ||
| raises=NotImplementedError, | ||
| reason=f"Parameterized types with tz={pa_dtype.tz} not supported.", | ||
| reason=f"Parameterized types {pa_dtype} not supported.", |
There was a problem hiding this comment.
#50689 fixes this for timestamptz, after both this and that are merged we can see about fixing this for decimal.
| @@ -4948,6 +4952,8 @@ def _get_engine_target(self) -> ArrayLike: | |||
| and not ( | |||
| isinstance(self._values, ArrowExtensionArray) | |||
| and is_numeric_dtype(self.dtype) | |||
| # Exclude decimal | |||
| and self.dtype.kind != "O" | |||
There was a problem hiding this comment.
can we be more specific than this? e.g. time32 would also go through here
There was a problem hiding this comment.
is_numeric_dtype(self.dtype) should block time32. I could import pyarrow and check pa.types.is_decimal if you prefer`
pandas/tests/extension/test_arrow.py
Outdated
| request.node.add_marker( | ||
| pytest.mark.xfail( | ||
| raises=NotImplementedError, | ||
| reason=f"pyarrow.type_for_alias cannot infer {pa_dtype}", |
There was a problem hiding this comment.
is this something we can patch in construct_from_string (OK for out of scope, just curious)
|
Greenish. Merging. |
|
Owee, I'm MrMeeseeks, Look at me. There seem to be a conflict, please backport manually. Here are approximate instructions:
And apply the correct labels and milestones. Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon! Remember to remove the If these instructions are inaccurate, feel free to suggest an improvement. |
* TST: Test ArrowExtensionArray with decimal types * Version compat * Add other xfails based on min version * fix test * fix typo * another typo * only for skipna * Add comment * Fix * undo comments * Bump version condition * Skip masked indexing engine for decimal * Some merge stuff * Remove imaginary test * Fix condition * Fix another test * Update condition (cherry picked from commit f5405b5)
…y with decimal types) (#51562)
closes #34166